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" The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )□ Responsive to comnnunication(s) filed on . 

2a)n This action is FINAL. 2b)S This action is non-final. 

3) 0 Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 
Disposition of Claims 

4) ^ Claim(s) 1-24 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) n Claim(s) is/are allowed. 

6) ^ Claim{s) 1:24 is/are rejected. 
?)□ Claim(s) is/are objected to. 

8) 0 C!aim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) n The specification is objected to by the Examiner. 

10)0 The drawing(s) filed on is/are: a)n accepted or b)\3 objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
1 !)□ The proposed drawing correction filed on is: a)n approved b)n disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) 0 The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

13) 0 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 

a)nAII b)n Some*c)n None of: 

1 0 Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) ^ Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 119(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121. 
Attachment(s) 

1 ) ^ Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-413) Paper No(s). . 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) □ Notice of Infomnal Patent Application (PTO-152) 

3) S Information Disclosure Statement(s) (PTO-1449) Paper No(s) 5 . 6) O Other: 



U.S. Patent and Trademarlt Office 
PTO-326 (Rev. 04-01) 



Office Action Summary 



Part of Paper No. 6 




Application/Control Number: 09/623,277 



Page 2 



Art Unit: 2172 



DETAILED ACTION 



Claims 1-24 are presented for examination. 



Priority 



2. Applicant's claim for domestic priority under 35 U.S.C. 1 19(e) is acknowledged. 



3. The references cited in the information disclosure statement, PTO-1449, have been fully 
considered. 



4. This application has been filed with informal drawings which are acceptable for 
examination purposes only. Formal drawings will be required when the application is allowed. 



5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 



Claims 1-24 are rejected under 35 U.S.C. 103(a) as being unpatentable over Thompson et 
al. (US Pat. No. 6,1 78,4 16)CThompson"). 

As per claims 1 and 13, Thompson teaches a method for generating a visual template for 
a concept (see col. 5, lines 30-33), as claimed comprises the steps of obtaining at least one initial 
query for the concept (thus, the user selects one or more of the instantiated query templates for 



Information Disclosure Statement 



Drawings 



Claim Rejections - 35 U.S. C § 103 
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further use as a query to the information sources that are stored in storage device 16, which is 
readable as obtaining at least one initial query for the concept)(see col. 5, lines 44-46); 

c. producing the additional query for inspection for appropriateness with respect to the 
concept (thus, the system advantageously presents the user with a listing of at least the most 
likely concepts and the user has the opportunity at step 140 to further refine his entry by selecting 
the most appropriate concept, which is readable as producing the additional query for inspection 
for appropriateness with respect to the concept)(see col. 5, lines 13-17); and 

d. in case of appropriateness, including the additional query in the visual template for the 
concept (thus, the instantiated templates are presented in a ranked order determined by 
pre-specified criteria, several possible criteria have been described the user selects one or more of 
the instantiated query templates for further use as a query to the information sources that are 
stored in storage device 16; which is readable as including the additional query in the visual 
template for the concept)(see col. 5, lines 41-46). But, Thompson does not explicitly indicate 
generating at least one additional query related to the initial query. However, Thompson 
indicates each template contains one or more typed variable, a query is then generated by 
entering into the system one or more keywords, each keyword is abstracted to a concept, each 
concept may be further refined by additional abstraction or by picking one concept from several 
candidates; which is readable as generating at least one additional query related to the initial 
query, (see col. 2, lines 31-37). Thus, it would have been obvious to a person of ordinary skill in 
the art at the time the invention was made to modify the teachings of Thompson with generating 
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at least one additional query related to the initial query. This modification would allow the 
teachings of Thompson to improve the accuracy and the reliability of the method and system for 
generating semantic visual templates for image and video retrieval, and provide access to 
relevant information (see col. 1, lines 19-20). 

As per claims 2 and 14, Thompson teaches a method as claimed, wherein each query is 
represented by an icon/example image (thus, sequences of characters, which is readable as 
icon/example image)(see col. 3, lines 39-42). 

As per claims 3 and 15, Thompson teaches a method as claimed, wherein the initial query 
is obtained via sketchpad (see col. 7, lines 5-18). 

As per claims 4 and 16, Thompson teaches a method as claimed, wherein generating the 
additional query comprises stepping a query feature with a step size which is inversely related to 
a weight associated with the query feature (thus, the presentation of altemative concepts is in 
ranked order where the ranking is determined by pre-specified criteria, one such criterion is 
fi-equency of selection during previous uses of the database dictionary; which is readable as 
wherein generating the additional query comprises stepping a query feature with a step size 
which is inversely related to a weight associated with the query feature)(see col. 5, lines 17-20). 

As per claims 5 and 17, Thompson teaches a method as claimed, wherein generating the 
additional icon comprises forming a join of plausible feature values (see col. 3, lines 34-42). 

As per claims 6 and 18, Thompson teaches a method as claimed, wherein appropriateness 
is ascertained by two-way user interaction (see col. 5, lines 6-10). 
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6. Claims 7-12 and 19-24 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Thompson et al. (US Pat. No. 6,178,416) in view of Hjelsvold et al. 'Searching and Browsing a 
Shared Video Database - 8/1995', ("Thompson"), ("Hjelsvold"). 

As per claims 7 and 19, in addition to the discussion in claim 1, Thompson teaches 
obtaining a textual query (thus, a textual presentation of the query template with variables, which 
is equivalent to textual query)(see col. 4, lines 12-14); 

c. using the visual attributes for forming a visual query (thus, query could be sent to a 
database with a relation named (druglnformation) that has at least the three attributes (drug), 
(symptom), and (effectiveness) the query would gather all information)(see col. 4, lines 57-59). 
Further, in column 2, lines 40 through 41, Thompson teaches the variables in the query templates 
are then instantiated with those concepts; 

d. using the visual query to retrieve information (thus, this query could be sent to a 
database with a relation named, the query would gather all information; which is readable as 
using the visual query to retrieve information)(see col. 4, lines 43-59). Further, in column 2, 
lines 44 through 45, Thompson teaches formulating queries to access any set of information 
sources; and 

e. displaying the information (thus, the presentation is made via a display, which is 
equivalent to displaying the information)(see col. 5, lines 37-43). But, Thompson does not 
explicitly indicate a video database. However, Hjelsvold indicates secondary context is a useful 
way to organise metadata in a video database that allows the users to define the degree of 
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information sharing during querying, (see page 97, col. 1, lines 12-15). Further, in page 90, lines 
32 through 33, Hjelsvold teaches a video database becomes even more complex because it has to 
support different users running different applications in sharing video. Thus, it would have been 
obvious to a person of ordinary skill in the art at the time the invention was made to modify the 
teachings of Thompson and Hjelsvold with a video database. This modification would allow the 
teachings of Thompson and Hjelsvold to improve the accuracy and the reliability of the method 
and system for generating semantic visual templates for image and video retrieval. 

As per claims 8 and 20, Thompson teaches a method as claimed, wherein the textual 
query is obtained from a keyboard (see col. 6, lines 18-21). 

As per claims 9 and 21, Thompson teaches a method as claimed, wherein the subset of 
the natural language comprises a small set of nouns, verbs, prepositions, adjectives and adverbs 
(thus, verbs, prepositions, adjectives and adverbs; equivalent to textual presentation) (see col. 4, 
lines 12-21). 

As per claims 10 and 22, Thompson teaches a method as claimed, further comprises a 
step of expanding the subset in an interactive fashion (see col. 2, lines 1-10). 

As per claims 1 1 and 23, in addition to the discussion in claim 9, Thompson teaches 
establishing a correspondence between the query and the natural language subset (thus, the 
concepts that are obtained the system finds all query templates that can use those concepts, the 
variables in the query templates are then instantiated with those concepts or with the keywords 
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used to form the concepts; which is readable as establishing a correspondence between the query 
and the natural language subset)(see col. 2, Hues 38-42); 

obtaining clarification if a word in the query is absent from the natural language subset 
and labeling the word accordingly (thus, the process may involve the successive entry of different 
keywords the abstraction of each keyword in turn and a selection from resulting concepts of the 
one concept that is deemed most appropriate; which is readable as obtaining clarification if a 
word in the query is absent from the natural language subset and labeling the word accordingly) 
(see col. 5, lines 25-29). Further, in column 7, lines 59 through 67, Thompson teaches the system 
searches the query templates for a possible match. 

As per claims 12 and 24, in addition to the discussion in claim 11, Thompson teaches 
with a the semantic visual template being a visual embodiment of a nouns in the query (thus, 
each query template that matches, instantiating the query template with the keywords or concepts 
that produced the semantic types that matched the template; which is readable as semantic visual 
template being a visual embodiment of a nouns in the query)(see col. 8, lines 52-54). 
7. The prior art made of record and not rehed upon is considered pertinent to applicant's 
disclosure. Kirk et al. US Patent Number 5,768,578 relates to information retrieval. Tong et al. 
RUBRIC - 'An Environment for Full Text Information Retrieval' relates to retrieval system. 
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Conclusion 

8. Any inquiry concerning this communication from examiner should be directed to Jean 
Bolte Fleurantin at (703) 308-6718. The examiner can normally be reached on Monday through 
Friday from 7:30 A.M. to 6:00 P.M. 

If any attempt to reach the examiner by telephone is unsuccessful, the examiner's 
supervisor, Mrs. KIM VU can be reached at (703) 305-8449. The FAX phone numbers for the 
Group 2100 Customer Service Center are: After Final (703) 746-7238, Official (703) 746-7239, 
and Non-Official (703) 746-7240. NOTE: Documents transmitted by facsimile will be entered 
as official documents on the file wrapper unless clearly marked "DRAFT\ 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the Group 2100 Customer Service Center receptionist whose telephone 
numbers are (703) 306-5631, (703) 306-5632, (703) 306-5633. 




Jean Bolte Fleurantin 
February 6, 2003 
JBF/ 



